The contributions of strategy use to working memory span : A comparison of strategy assessment

In two experiments, we tested whether individual differences in strategy production account for individual differences in performance on a working memory span task. We measured the strategies used during a standard experimenter-paced operation span (OSPAN) task by having participants make both set-by-set reports of strategy use for individual item sets and global reports of strategy use. In Experiment 1, although normatively effective strategies were self-reported on only a small proportion of OSPAN sets, individual differences in effective strategy use correlated with span performance. Experiment 2 replicated this outcome using a sample of 100 participants but, as important, it demonstrated that individual differences in effective strategy use did not mediate the relationship between OSPAN and measures of verbal ability. Discussion focuses on the interpretation of strategy–span relationships and the relative utility of general reports of strategy use versus the set-by-set reports introduced here for the OSPAN task. Article: What does performance on a working memory (WM) span task measure, and why do individual differences in span often correlate with measures of higher order cognition, such as reading comprehension and inductive reasoning? These questions have been central to two decades of theoretical and applied research about working memory. Answers arise from prominent theories of WM, most of which conceptualize individual differences in WM span as reflecting a general process or mechanism that influences the efficiency of the WM system. For instance, Hasher and colleagues argue that individual differences in span reflect the degree to which people inhibit irrelevant, or no-longer-relevant, information from WM when performing the task (e.g., Lustig, May, & Hasher, 2001; May, Hasher, & Kane, 1999). In contrast, Engle and his colleagues suggest that variability in span performance derives from individual differences in attention control capabilities that are critical to both active maintenance and inhibition (e.g., Engle & Kane, 2004; Engle, Kane, & Tuholski, 1999). Other generalprocess theories of WM are available (Miyake & Shah, 1999; see also Cowan, 2005; Oberauer, 2005), but importantly, they all emphasize that WM capacity differences—as measured by WM span—largely reflect the integrity of general processes or mechanisms that are central to the WM system (Conway, Kane, & Engle, 2003). Thus, the Zeitgeist of WM theory is echoed in Earl Hunt’s (1999) observation that ―the concept of memory span is clearly a system architecture concept‖ (p. 29). Our aim is not to competitively evaluate general theories of WM architecture or mechanism. Instead, we critically examine an alternative class of hypotheses that has recently been forwarded. According to this alternative, differences in span performance are assumed to result from variability in the algorithms, or strategies, that individuals use while performing span tasks; moreover, the power of WM span tasks to predict variation in complex cognitive abilities may also derive from individual differences in strategy use (e.g., Baddeley, 2000; McNamara & Scott, 2001; Salthouse, 1996). To illustrate how algorithmic hypotheses differ from architectural hypotheses, consider how they might explain individual differences in the operation span (OSPAN) task, a frequently used measure of WM capacity. In a typical administration of OSPAN, a mathematical operation and a word appear on a computer screen—for example, Is (9/3) – 2 = 2? aunt—and a participant must read the problem aloud—―Is (9/3) – 2 = 2?‖—then verify the answer as quickly and accurately as possible (―no‖), and then immediately read the word aloud (―aunt‖). At that time, the experimenter quickly advances the screen to the next operation–word pair. These operation– word pairs are individually presented in sets that usually range from 2 to 6 pairs per set. After the last operation–word pair of a set is presented, the participant is prompted to recall the words from that set in the order that they had appeared. Although participants need to process the operations quickly and accurately, they are instructed that their primary goal is to remember the words in serial order. Performance on this span task can be measured in a variety of ways, but it is almost always based on the number of words correctly recalled (Conway et al., 2005; but see Waters & Caplan, 1996). The substantial individual differences that arise in WM span are readily explained in terms of variability in some relatively general mechanism but, in principle, they can be equally well accounted for by differences in strategic behaviour. Such an algorithmic, strategy mediation hypothesis explains individual differences in OSPAN in terms of the strategies that people use to learn and recall the word sets. Some strategies are likely to be relatively ineffective, such as passively reading each word, whereas others are likely to be more effective, such as using interactive imagery or sentences to associate the words within a set. By the strategy mediation hypothesis, individuals who perform best in span tasks are those who use more effective strategies (e.g., McNamara & Scott, 2001). The present article describes the outcomes of two experiments involving the OSPAN task that empirically evaluate a core prediction of the strategy mediation hypothesis—namely, that individual differences in strategy use predict span performance. In doing so, we introduce a new measure of strategy use for the OSPAN task based on set-by-set reports and also compare it to a traditional measure of strategy use based on general retrospective reports. Review of research relevant to the strategy mediation hypothesis At least two lines of research bear on the strategy mediation hypothesis and motivated the approach that we adopted to test it. First, some studies have investigated whether instructions to use a particular strategy, such as imagery or rehearsal, can influence span (e.g., McNamara & Scott, 2001; Turley-Ames & Whitfield, 2003). For instance, Turley-Ames and Whitfield compared performance on the OSPAN task immediately prior to, and following, strategy instruction. Strategy-instructed participants were told to use one of several mnemonics to remember the target words, such as repeatedly rehearsing the words aloud, creating a sentence or story from the words, or generating visual images of the words. Such strategy instructions, as well as more intensive training in the use of a particular strategy (McNamara & Scott, 2001), significantly boosted span scores. A second line of work used self-paced viewing times to infer the contribution of strategy use to span performance. Engle, Cantor, and Carullo (1992) and Friedman and Miyake (2004) examined the time that participants allocated to various components of span tasks, such as viewing each to-be-remembered word, as an indication of the ways in which participants strategically approached the task. For instance, Engle et al. (1992, Exp. 1) allowed participants unlimited time to examine the different components of each operation–word pair (with these components presented via a self-paced, moving-window technique) in OSPAN. Viewing time on the to-be-remembered words correlated significantly with span performance (r = .29), suggesting that high-WM individuals read (or studied, or rehearsed) the target words for more time than did low-WM individuals. Thus, high spans appeared more strategic than did low spans when performing the OSPAN task. One limitation of this conclusion is that viewing times are an indirect measure of strategy production and may be influenced by a host of other factors that, in principle, could account for the aforementioned results (McNamara & Scott, 2001). Moreover, time spent on a task will only reflect strategic behaviour to the degree that different strategies result in different time signatures, which may not be the case for many mediational strategies, such as imagery, sentence generation, and rehearsal, which participants may use when attempting to maintain the to-beremembered words. Thus, although research on viewing times has led to important insight into WM theory, this research does not directly indicate how often mediational strategies are used during standard WM span tasks. Our goal was to evaluate directly whether strategy use influences WM span performance when the span task is conducted using a standard procedure that involves no strategy training and minimizes strategy use by allowing participants only limited time to read each to-be-remembered word (in contrast to the self-paced, movingwindow technique). An assessment of strategy use in previous studies typically has involved using working memory tasks that increase the likelihood of strategy use, in contrast to the standard tasks often used in the literature to provide a more process-pure measure of WM ability. Accordingly, previous research aimed at evaluating the strategy mediation hypothesis may not generalize to the standard preparations used by the field, a limitation that we sought to circumvent in the present research. In particular, rather than allowing participants to pace themselves through the task, in our studies (as in most) the experimenter controlled the stimulus presentation. Because the experimenter-driven procedure presumably minimizes strategic behaviour (e.g., see Friedman & Miyake, 2004, Appendix), it arguably yields the most conservative test of the strategy mediation hypothesis, and the one most relevant to research using the standard, experimenter-controlled procedure. 1 Of course, to estimate the degree to which strategies affect span scores in experimenter-paced procedures, we must obtain a measure of strategy use other than reading time. One obvious candidate is to simply ask participants, retrospectively, about the strategies that they had used to accomplish the task. This is precisely how prior studies on WM span have assessed strategy use (e.g., Friedman & Miyake, 2004; Harris & Qualls, 2000; McNamara & Scott, 2001; Turley-Ames & Whitfield, 2003). After all trials of the span task were completed, participants explained in general how they tried to complete the task. These general retrospective reports have yielded some support for the strategy mediation hypothesis. For instance, Turley (1997, cited in Turley-Ames & Whitfield, 2003) found that many participants reported rehearsing the to-be-remember words in the OSPAN task, with more high-span than low-span participants reporting such strategy use (69% vs. 24%, respectively). Friedman and Miyake (2004) found, in their experimenter-controlled reading span task, that 59% of their participants reported using some phonological strategy at some point during a reading span task, whereas 45% and 18% reported using a semantic-based and imagery-based strategy, respectively. Moreover, participants who used the imagery-based strategy earned higher span scores than did those who did not. One interpretation of these outcomes is that strategy use accounts for some of the individual differences in WM span, which supports the strategy mediation hypothesis. Even so, conclusions based on retrospective general reports are open to alternative interpretations. First, given that the reports are general, participants may develop the report in ways that do not accurately reflect how the task was actually performed (for detailed discussion, see Ericsson & Simon, 1984). For example, participants may draw on beliefs about how they ought to have performed the task, or they may remember how they performed a few trials of the task and generalize these particular instances to the remaining trials. Second, given that the reports are retrospective, participants may simply forget how they performed many of the individual trials. Of utmost importance here, individual differences may occur in any of these processes or biases, and they may or may not be associated with WM capacity. For instance, if low-span individuals are particularly susceptible to forgetting prior to making the general retrospective reports (see Kane & Engle, 2000), then any span/strategy correlation may not reflect true differences in strategy production per se. Retrospective general reports therefore do not provide definitive evidence about the strategy mediation hypothesis, namely that strategy use accounts for variation in WM span. Current approach to evaluating the strategy mediation hypothesis Our approach involved having participants make set-by-set reports of strategy use for the OSPAN task. In particular, sometime after a participant attempted to recall the to-be-remembered words of a given set, he or she reported the particular strategy—if any—used to remember the words for that set. This report was based on a specific probe that assessed the strategies that pilot work and prior research (e.g., Turley-Ames & Whitfield, 2003) suggested are sometimes used during the OSPAN task. Prompt used to obtain set-by-set strategy reports. For concurrent reports (Experiment 1), the prompt for the report was presented immediately after a participant attempted recall for the words within a set. For retrospective reports (Experiments 1 and 2), all sets were presented for the OSPAN task first. Afterwards, each set was individually presented in its entirety above this prompt for the strategy report. How did you originally try to remember the words from the series above? State the number corresponding to the most appropriate response: 1 = read each word as it appeared 2 = repeated the words as much as possible 3 = used a sentence to link the words together 4 = developed mental images of the words 5 = grouped the words in a meaningful way 6 = did something else These set-by-set reports demonstrate acceptable validity and have minimal reactive effects on task processing when they are used to measure strategy use in related domains, such as in associative learning and free recall (Dunlosky & Hertzog, 2001; Hertzog, McGuire, & Lineweaver, 1998). Even so, given that the present research is the first to adapt these reports to WM span, we report evidence relevant to their key properties in Experiment 1. 2 As important, set-by-set reports of strategy use are superior to general reports in numerous ways (Ericsson & Simon, 1980), but here we consider only one. Imagine that in response to a general retrospective prompt, two participants report that they used rehearsal and interactive imagery to complete the OSPAN task. However, one participant actually used imagery on 80% of the sets, whereas the other did on only 20%. This critical difference would not be reflected in the general reports, but it would be in set-by-set reports. 3 In this case, set-by-set reports would provide a more accurate description of strategy use at the level of individual sets, which itself is necessary for conducting fine-grained analyses conditionalized on set. Note, however, that our aim is not to dismiss entirely the potential benefit of general reports, which we collected in both experiments. Doing so not only afforded tests of the main prediction based on converging operations, but also allowed us to directly compare the two kinds of strategy report. To foreshadow, our results supported conclusions from earlier research that had exclusively used general reports of strategy use, but they also demonstrated some of the superior characteristics of set-by-set reports. In summary, we evaluated a prediction of the strategy mediation hypothesis by examining the relations among individual differences in the measures of strategy use and in OSPAN performance. If strategy use accounts for individual differences in span performance, then the correlation between the use of effective strategies and span performance will be positive. EXPERIMENT 1 Given that set-by-set strategy reports are new to investigations of span tasks, we used both concur-rent reports and retrospective reports. The former were made immediately after a participant attempted to recall the target words of each set, whereas the latter were made after a participant had completed all the sets of the OSPAN task. Thus, both general reports and retrospective set-by-set reports (hereafter, referred to as retrospective reports) occurred after the OSPAN task had been completed, but the former asked for a general report of strategy use across all sets whereas the latter involved reporting the strategy used for each set individually. To prompt the retrospective reports, each original set was re-presented individually, with all the equations and words from the set presented together. By using both kinds of set-by-set report, we could deter-mine (a) whether they demonstrated acceptable validity, (b) whether concurrent reports had reactive effects on task performance, and (c) whether retrospective reports accurately captured the strategies that participants reported using at study (i.e., whether retrospective reports suffered from forgetting). To address these issues, we used methods from Dunlosky and Hertzog (2001), who demonstrated the validity of concurrent and retrospective strategy reports during associative learning. In Experiment 1, two groups of participants performed the standard OSPAN task, and both made retrospective strategy reports. The retrospective-only group made only retrospective reports, whereas the concurrent + retro group made both kinds of strategy report. If the reports have construct validity, then OSPAN performance will be better for trials in which participants reported using normatively more effective strategies (e.g., imagery) than normatively less effective strategies (e.g., reading). If concurrent reports have reactive effects by influencing the use of strategies, then the profile of retrospective reports will differ for the concurrent + retro group and retrospective-only group. Finally, if the validity of retrospective reports is reduced due to forgetting, then the profile of reported strategies will differ markedly when the reports are made concurrently versus retrospectively. Importantly, we found that the reports demonstrated acceptable validity and were quite consistent for concurrent and retrospective reports. Thus, to achieve greater statistical power for analyses relevant to the strategy mediation hypothesis, we collapsed across the two groups and analysed retrospective reports where appropriate.